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1 Definitions 


A matrix (plural: matrices) is simply a rectangular array of “things”. For now, we’ll assume the “things” are numbers, 
but as you go on in mathematics, you’1l find that matrices can be arrays of very general objects. Pretty much all that’s 
required is that you be able to add, subtract, and multiply the “things”. 


Here are some examples of matrices. Notice that it is sometimes useful to have variables as entries, as long as the 
variables represent the same sorts of “things” as appear in the other slots. In our examples, we’ll always assume that 
all the slots are filled with numbers. All our examples contain only real numbers, but matrices of complex numbers 
are very common. 
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The first example is a square 3 X 3 matrix; the next is a 2 x 4 matrix (2 rows and 4 columns—if we talk about a matrix 
that is “m x n” we mean it has m rows and n columns). The final two examples consist of a single column matrix, 
and a single row matrix. These final two examples are often called “vectors”—the first is called a “column vector” and 
the second, a “row vector’. We’ll use only column vectors in this introduction. 


Often we are interested in representing a general m x n matrix with variables in every location, and that is usually 
done as follows: 


@1 G2 13 Gin 
G21 G22 493 Q2n 
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The number in row 7 and column 7 is represented by a;;, where 1 <7 < mand1 < 7 < n. Sometimes when there is 
no question about the dimensions of a matrix, the entire matrix can simply be referred to as: 


(aij) . 


1.1. Addition and Subtraction of Matrices 


As long as you can add and subtract the “things” in your matrices, you can add and subtract the matrices themselves. 
The addition and subtraction occurs in the obvious way—element by element. Here are a couple of examples: 


i, ak 47 9 2.4 4 5 8 
2 6 -4)4/55 3 -e]=[75 9 —-4-e 
215 « 2 5 V2 4 20 r+vV2 


13 #7 2 ee —2 1 6 
2 6 -4)—-[55 3 -e]l=|/[-35 3 e-4 
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To find what goes in row 7 and column 9 of the sum or difference, just add or subtract the entries in row 7 and column 
j of the matrices being added or subtracted. 


In order to make sense, both of the matrices in the sum or difference must have the same number of rows and columns. 
It makes no sense, for example, to add a 2 x 4 matrix toa 3 x 4 matrix. 


1.2 Multiplication of Matrices 


When you add or subtract matrices, the two matrices that you add or subtract must have the same number of rows and 
the same number of columns. In other words, both must have the same shape. 


For matrix multiplication, all that is required is that the number of columns of the first matrix be the same as the 
number of rows of the second matrix. In other words, you can multiply an m x k matrix by a & Xx n matrix, with 
the m x k& matrix on the left and the k x nm matrix on the right. The example on the left below represents a legal 
multiplication since there are three columns in the left multiplicand and three rows in the right one; the example on the 
right doesn’t make sense—the left matrix has three columns, but the right one has only 2 rows. If the matrices on the 
right were written in the reverse order with the 2 x 3 martix on the left, it would represent a valid matrix multiplication. 
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So now we know what shapes of matrices it is legal to multiply, but how do we do the actual multiplication? Here is 
the method: 


If we are multiplying an m x k matrix by a k x n matrix, the result will be an m x n matrix. The element in the 
product in row ¢ and column 7 is gotten by multiplying termwise all the elements in row ¢ of the matrix on the left by 
all the elements in column 7 of the matrix on the right and adding them together. 


Here is an example: 


1 3 2 4 11 32-59 
5 0 7 6 10] =] 55 118 
6 9 8 5.6 (9 118 228 


To find what goes in the first row and first column of the product, take the number from the first row of the matrix on 
the left: (1, 3,2), and multiply them, in order, by the numbers in the first column of the matrix on the right: (4,6, 5). 
Add the results: 1-4+3-6+2-5=4+18+ 10 = 32. To get the 228 in the third row and second column of the 
product, the use the numbers in the third row of the left matrix: (6, 9, 8) and the numbers in the second column of the 
right matrix: (11, 10,9) to get6-11+9-10+8-9=66+4+ 90+ 72 = 228. 

Check your understanding by verifying that the other elements in the product matrix are correct. 


In general, if we multiply a general m x & matrix by a general & x n matrix to get an m xX n matrix as follows: 


Q@11 G2 ‘**  Gik bit big ss: Din C11 Cig *t' Ein 
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Then we can write c;; (the number in row 7, column 7) as: 
k 
cig = > aipdy;. 
p=1 


1.3. Square Matrices and Column Vectors 


Although everything above has been stated in terms of general rectangular matrices, for the rest of this tutorial, we'll 
consider only two kinds of matrices (but of any dimension): square matrices, where the number of rows is equal to 
the number of columns, and column matrices, where there is only one column. These column matrices are often 
called “vectors”, and there are many applications where they correspond exactly to what you commonly use as sets of 
coordinates for points in space. In the two-dimensional x-y plane, the coordinates (1, 3) represent a point that is one 
unit to the right of the origin (in the direction of the z-axis), and three units above the origin (in the direction of the 
y-axis). That same point can be written as the following column vector: 


1 
3) ° 
If you wish to work in three dimensions, you’ll need three coordinates to locate a point relative to the (three- 


dimensional) origin—an z-coordinate, a y-coordinate, and a z-coordinate. So the point you’d normally write as 
(x, y, z) can be represented by the column vector: 


Quite often we will work with a combination of square matrices and column matrices, and in that case, if the square 
matrix has dimensions n x n, the column vectors will have dimension n x 1 (n rows and | column)!. 


1.4 Properties of Matrix Arithmetic 


Matrix arithmetic (matrix addition, subtraction, and multiplication) satisfies many, but not all of the properties of 
normal arithmetic that you are used to. All of the properties below can be formally proved, and it’s not too difficult, 
but we will not do so here. In what follows, we’ll assume that different matrices are represented by upper-case letters: 
M,N, P,..., and that column vectors are represented by lower-case letters: v,w,.... 


We will further assume that all the matrices are square matrices or column vectors, and that all are the same size, either 
nm xX norn X 1. Further, we’ll assume that the matrices contain numbers (real or complex). Most of the properties 
listed below apply equally well to non-square matrices, assuming that the dimensions make the various multiplications 
and addtions/subtractions valid. 


Perhaps the first thing to notice is that we can always multiply two n x nm matrices, and we can multiply ann x n 
matrix by a column vector, but we cannot multiply a column vector by the matrix, nor a column vector by another. In 
other words, of the three matrix multiplications below, only the first one makes sense. Be sure you understand why. 


1 2 3 10 10 1 2 3 10 7 
4 5 6 11 11 4 5 6 11 8 
7 8 9 12 12 7 8 9 12 9 


'We could equally well use row vectors to correspond to coordinates, and this convention is used in many places. However, the use of column 
matrices for vectors is more common 


Finally, an extremely useful matrix is called the “identity matrix”, and it is a square matrix that is filled with zeroes 
except for ones in the diagonal elements (having the same row and column number). Here, for example, is the 4 x 4 
identity matrix: 


1 0 0 0 
0 1 0 0 
001 0 
0001 


The identity matrix is usually called “F’’ for any size square matrix. Usually you can tell the dimensions of the identity 
matrix from the surrounding context. 


e Associative laws: 


(MN)P = M(NP) (MN)v = M(Nv) 
(M+N)+P=M+(N+P) (utv)+tw=ut(vt+w) 


e Commutative laws for addition: 
M+N=N+M v+tw=wty 
e Distributive laws: 


M(N+£P)=MNitMP (M4N)P=MPitNP 
M(vtw)=Mvti Mw (M+ N)v = MviNv 


e The identity matrix: 
NI=IN=N Iv=v 


Probably the most important thing to notice about the laws above is one that’s missing—multiplication of matrices is 
not in general commutative. It is easy to find examples of matrices M and N where MN #4 NM. In fact, matrices 
almost never commute under multiplication. Here’s an example of a pair that don’t: 


1 o1\ fl 0) f2 1). 1 O0O\ fl 1\_f1ii1 

0 1/y\1 TY \ 1’ 1 y\o ly 2° 
So the order of multiplication is very important; that’s why you may have noticed the care that has been taken so far 
in describing multiplication of matrices in terms of “the matrix on the left’, and “the matrix on the right”. 


The associative laws above are extremely useful, and to take one simple example, consider computer graphics. As 
we’ ll see later, operations like rotation, translation, scaling, perspective, and so on, can all be represented by a matrix 
multiplication. Suppose you wish to rotate all the vectors in your drawing and then to translate the results. Suppose 
R and T are the rotation and translation matrices that do these jobs. If your picture has a million points in it, you 
can take each of those million points y and rotate them, calculating Rv for each vector v. Then, the result of that 
rotation can be translated: T'(Rv), so in total, there are two million matrix multiplications to make your picture. But 
the associative law tells us we can just multiply T' by R once to get the matrix T'R, and then multiply all million points 
by TR to get (T'R)v, so all in all, there are only 1,000,001 matrix multiplications— one to produce TR and a million 
multiplications of TR by the individual vectors. That’s quite a savings of time. 


The other thing to notice is that the identity matrix behaves just like 1 under multiplication—if you multiply any 
number by 1, it is unchanged; if you multiply any matrix by the identity matrix, it is unchanged. 


1.5 Matrix Inversion 


We noted that the identity matrix, J, has the property that if you multiply any other matrix by it (either on the right or 
on the left), the result is the original matrix. It’s just like multiplying a number by 1—it leaves the number unchanged. 


In the arithmetic of real numbers, we have the concept of an inverse that in some sense “undoes” a multiplication. If 
I multiply a number by 5, I can undo that multiplication with another multiplication by 1/5. We sometimes say that 5 
and 1/5 are multiplicative inverses of each other. In this section we’ll see that something similar can take place with 
(some) matrices. 


As a concrete example, we’ll look at the solution to a system of linear equations. 


1.6 Systems of Linear Equations 


Let’s take a look at a problem that may seem a bit boring, but in terms of practical applications is perhaps the most 
common use of matrices: the solution of systems of linear equations. Following is a typical problem (although real- 
world problems may have hundreds of variables). 


Solve the following system of equations: 


gr+4y+3z = 7 
2e+5y+4z = 11 
ge-3sy—2z = 5. 


The key observation is this: the problem above can be converted to matrix notation as follows: 


1 4 3 x 7 
2 5 4 y|=]1l{. (1) 
1 -3 -2 z 5 


The numbers in the square matrix are just the coefficients of x, y, and z in the system of equations. Check to see that 
the two forms—the matrix form and the system of equations form—trepresent exactly the same problem. 


Ignoring all the difficult details, here is how such systems can be solved. Let M be the 3 x 3 square matrix in 
equation (1) above, so the equation looks like this: 


x 7 
Miy)={1). (2) 
iZ 5 


Suppose we can somehow find another matrix N such that NAZ = I. If we can, we can multiply both sides of 
equation (2) by N to obtain: 


x x x 7 
NM ly] =lIlyJ=lyJ=N I 11], 
Zz z z 5 


so we can simply multiply our matrix N by the column matrix containing the numbers 7, 11, and 5 to get our solution. 


Without explaining where we got it, the matrix on the left below is just such a matrix N. Check that the multiplication 
below does yield the identity matrix: 


2 -l 1 1 4 3 1 
8 -5 2 2 5 4 ]=1]0 
-11 7 -3 1 -3 -2 0 


So we just need to multiply that matrix N by the column vector containing 7, 11, and 5 to get our solution: 


2 -l 1 7 8 x 
8 -5 2 11}=] 11 J=tly 
-11 7 -3 i) —15 y 
From this last equation we conclude that z = 8, y = 11, and zg = —15is a solution to the original system of equations. 


You can plug them in to check that they do indeed form a solution. 


Although it doesn’t happen all that often, somtimes the same system of equations needs to be solved for a variety 
of column vectors on the right—not just one. In that case, the solution to every one can be obtained by a single 
multiplication by the matrix N. 


The matrix N is usually written as M—!, called “M-inverse’. It is a multiplicative inverse in just the same way that 
1/3 is the inverse of 3: 3- (1/3) = 1, and 1 is the multiplicative identity, just as J is in matrix multiplication. Entire 
books are written that describe methods of finding the inverse of a matrix, so we won’t go into that here. 


Remember that for numbers, zero has no inverse; for matrices, it is much worse—many, many matrices do not have 
an inverse. Matrices without inverses are called “singular”. Those with an inverse are called “non-singular”. 


Just as an example, the matrix on the left of the multiplication below can’t possibly have an inverse, as we can see 
from the matrix on the right. No matter what the values are of a, b,... ,4, itis impossible to get anything but zeroes in 
certain spots in the diagonal, and we need ones in all the diagonal spots: 


1 0 0 a be a be 
00 0 de fJ= [0 0 0 
0 0 0 g h i 0 0 0 


If the set of linear equations has no solution, then it will be impossible to invert the associated matrix. For example, 
the following system of equations cannot possibly have a solution, since x + y + z cannot possibly add to two different 
numbers (7 and 11) as would be required by the first two equations: 


et+yt+z = 7 
et+yt+z = il 
g-3y-—2z = 5. 


So obviously the associated matrix below cannot be inverted: 


1 oil 1 
1 oil 1 
1 -3 -2 


It is also true (but we won’t bother to prove it) that if Mf and M/—! are a matrix and its inverse, then: 
MM-+=M'M =I. 


In other words, the “left inverse” is the same as the “right inverse”. 


2 Computer Graphics and Linear Transformations 


Some computers can only draw straight lines on the screen, but complicated drawings can be made with a long series 
of line-drawing instructions. For example, the letter “F” could be drawn in its normal orientation at the origin with the 
following set of instructions: 


1. Draw a line from (0, 0) to (0, 5). 
2. Draw a line from (0, 5) to (3, 5). 
3. Draw a line from (0, 3) to (2, 3). 


Imagine that you have a drawing that’s far more complicated than the “F” above consisting of thousands of instructions 
similar to those above. Let’s take a look at the following sorts of problems: 


1. How would you convert the coordinates so that the drawing would be twice as big? How about stretched twice 
as high (y-direction) and three times as wide (z-direction)? 


2. Could you draw the mirror image through the y-axis? 
3. How would you shift the drawing 4 units to the right and 5 units down? 


4. Could you rotate it 90° counter-clockwise about the origin? Could you rotate it by an angle @ counter-clockwise 
about the origin? 


5. Could you rotate it by an angle @ about the point (7, —3)? 


6. Ignoring the problem of making the drawing on the screen, what if your “drawing” were in three dimensions? 
Could you solve problems similar to those above to find the new (3-dimensional) coordinates after your object 
has been translated, scaled, rotated, et cetera? 


2.1 Scaling, Translation and Rotation 


It turns out that the answers to all of the problems above can be achieved by multiplying your vectors by a matrix. Of 
course a different matrix will solve each one. Here are the solutions: 
Graphics Solution 1: 


To scale in the z-direction by a factor of 2, we need to multiply all the x coordinates by 2. To scale in the y-direction, 
we similarly need to multiply the y coordinates by the same scale factor of 2. The solution to scale any drawing by a 
factor sz in the x-direction and s, in the y-direction is to multiply all the input vectors by a general scaling matrix as 


follows: 
(5 2) Go) = Gi) 
O sys \y Sy) 
To uniformly scale everything to twice as big, let s; = s, = 2. To scale by a factor of 2 in the x-direction and 3 in the 
y-direction, let s, = 2 and sy = 3. 


We’ll illustrate the general procedure with the drawing instructions for the “F”’ that appeared earlier. The drawing 
commands are described in terms of a few points: (0,0), (0,5), (3,5), (0,3), and (2,3). If we write all five of those 
points as column vectors and multiply all five by the same matrix (either of the two above), we’ll get five new sets 
of coordinates for the points. For example, in the case of the second example where the scaling is 2 times in # and 3 
times in y, the five points will be converted by matrix multiplication to: (0,0), (0,15), (6,15), (0,9) and (4,9). If we 
rewrite the drawing instructions using these transformed points, we get: 


1. Draw a line from (0, 0) to (0, 15). 
2. Draw a line from (0, 15) to (6, 15). 
3. Draw a line from (0, 9) to (6, 9). 


Follow the instructions above and see that you draw an appropriately stretched “F’’. In fact, do the same thing for each 
of the matrix solutions in this set to verify that the drawing is transformed appropriately. Notice that if s, or sy is 
smaller than 1, the drawing will be shrunk—not expanded. 

Graphics Solution 2: 


A mirror image is just a scaling by —1. To mirror through the y-axis means that each z-coordinate will be replaced 
with its negative. Here’s a matrix multiplication that will do the job: 


(o 3) = Ga). 


To translate points 4 to the right and 5 units down, you essentially need to add 4 to every x coordiante and to subtract 
5 from every y coordinate. If you try to solve this exactly as in the examples above, you'll find it is impossible. To 
convince yourself it’s impossible with any 2 x 2 matrix, consider what will happen to the origin: (0,0). You want to 
move it to (4, —5), but look what happens if you multiply it by any 2 x 2 matrix (a, b, c, and d can be any numbers): 


a b\ f0\_ /0 

c dj] \0/ \O/° 
In other words, no matter what a, b,c, and d are, the matrix will map the origin back to the origin, so translation using 
this scheme is impossible. 


Graphics Solution 3: 


But there’s a great trick”. For every one of your two-dimensional vectors, add an artificial third component of 1. So the 
point (3, 6) will become (3, 6, 1), the origin will become (0,0, 1), et cetera. The column vectors will now have three 
rows, so the transformation matrix will need to be 3 x 3. To translate by ¢, in the x-direction and ¢, in the y-direction, 
multiply the artifically-enlarged vector by a matrix as follows: 


10 £ E+ ty 
Olt yl =l[ytty 
00 1 1 1 


The resulting vector is just what you want, and it also has the artificial 1 on the end that you can just throw away. To 
get the particular solution to the problem proposed above, simply let tz = 4 and ty = —5. 


But now you're probably thinking, “That’s a neat trick, but what happens to the matrices we had for scaling? What a 
pain to have to convert to the artificial 3-dimensional form and back if we need to mix scaling and translation.” The 
nice thing is that we can always use the artifically extended form. Just use a slightly different form of the scaling 
matrix: 


8s, 0 0 x S20 
0 sy, O y | = | syy 
0 O 1 1 1 


In the solutions that follow, we’ll always add a 1 as the artifical third component?. 
Graphics Solution 4: 


Convince yourself (by drawing a few examples, if necessary) that to rotate a point counter-clockwise by 90° about the 
origin, you will basically make the original x coordinate into a y coordinate, and vice-versa. But not quite. Anything 
that had a positive y coordinate will, after rotation by 90°, have a negative x coordinate and vice-versa. In other words, 


7In fact, it’s a lot more than a trick—it is really part of projective geometry. 
3But in the world of computer graphics or projective geometry, it is often useful to allow values other than 1—in perspective transformations, 
for example 


the new y coordinate is the old x coordinate, and the new z coordinate is the negative of the old y coordinate. Convince 
yourself that the following matrix does the trick (and notice that we’ve added the | as an artificial third component): 


0 -1 O x -y 
1 0 O yj=|ez 
0 0 1 1 1 


The general solution for a rotation counter-clockwise by an angle @ is given by the following matrix multiplication: 


cos@ —sin@ 0 x xcos@ — ysind 
sin? cos@ 0 y | = | zsin@+ycosé 
0 0 1 1 1 


If you’ve never seen anything like this before, you might consider trying it for a couple of simple angles, like @ = 45° 
or 9 = 30° and put in the drawing coordinates for the letter “F” given earlier to see that it’s transformed properly. 


Graphics Solution 5: 


Here is where the power of matrices really comes through. Rather than solve the problem from scratch as we have 
above, let’s just solve it using the information we already have. Why not translate the point (7, —3) to the origin, 
then do a rotation about the origin, and finally, translate the result back to (7, —3)? Each of those operations can be 
achieved by a matrix multiplication. Here is the final solution: 


10 7 cos@ —sin@ 0 10 -7 x 
01 -3 sind cos@ 0O 01 3 y 
00 1 0 0 1 00 1 1 


Notice carefully the order of the matrix multiplication. The matrix closest to the (2, y, 1) column vector is the first one 
that’s applied to it—it should move (7, —3) to the origin. To do that, we need to translate x coordinates by —7 and y 
coordinates by 3. The next operation to be done is the rotation by an arbitrary angle @, using the matrix form from the 
previous problem. Finally, to translate back to (7, —3) we have to translate in the opposite direction from what we did 
originally, and the matrix on the far left above does just that. 


Remember that for any particular value of @, sin @ and cos @ are just numbers, so if you knew the exact rotation angle, 
you could just plug the numbers in to the middle 3 x 3 matrix and multiply together the three matrices on the left. 
Then to transform any particular point, there would be only one matrix multiplication involved. 


To convince yourself that we’ve got the right answer, why not put in a particular (simple) rotation of 90° into the 
matrices and work it out? cos 90° = 0 and sin 90° = 1, so the product of the three matrices on the left is: 


10 7 0 -l1 0 1 0 -7 0 -1 4 
0 1 -3 1 0 0 01 3}]=]1 O -10 
0 0 1 0 0 1 00 1 0 0 1 


Try multiplying all the vectors from the “F” example by the single matrix on the right above and convince yourself 
that you’ve succeeded in rotating the “F” by 90° counter-clockwise about the point (7, —3). 


Graphics Solution 6: 


The answer is yes. Of course you'll have to add an artificial fourth dimension which is always 1 to your three- 
dimensional coordinates, but the form of the matrices will be similar. 


On the left below is the mechanism for scaling by sz, sy, and s, in the z-, y-, and z-directions; on the right is a 
multiplication that translates by ¢,, ¢,, and £, in the three directions. 


S&S O 0 O x 100 x 
0 sy O O y 0104, y 
0 0 s, O z 001 £, z 
0 0 0 1 1 0 0 0 1 


Finally, to rotate by an angle of @ counter-clockwise about the three axes, multiply your vector on the left by the 
appropriate one of the following three matrices (left, middle, and right correspond to rotation about the z-axis, y-axis, 
and z-axis: 


1 0 0 0 cos@ 0 sin@d O cos@ —sind 0 0 
0 cos@ —sin@ 0 0 1 0 0 sin@ cos? O 0 
0 sin? cos@ O —sin? 0 cosé 0 0 0 1 0 
0 O 0 1 0 0 oOo 1 0 0 01 


2.2. Inverses of Transformations 


Almost all the operations above can be “undone”, and the matrix operation that undoes them, not surprisingly, is the 
inverse of the matrix operation that “does” them. Let’s take a simple example—translation. We know that the matrix 
that translates by a distance tz in the x-direction and by ¢, in the y-direction is this: 


8 


t 
t 


i~9 


1 0 
0 1 
0 0 


e 


From a geometric viewpoint, to undo a translation, we simply need to translate in the opposite direction—namely by 
—t, and —t, in the z- and y-directions. As an exercise, show that in fact: 


1 0 te\ [1 0 —te 10 0 
01 t,){0 1 -t,)=]0 1 0 
00 1/\o 0 1 001 


As long as neither of the scale factors sz and s, are zero, we can undo a scaling transformation with another scaling 
matrix having factors 1/sz and 1/s,. To undo a rotation by an angle @, simply rotate by —9. 
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